Papers with speaker models
Unified Pragmatic Models for Generating and Following Instructions (N18-1)
Copied to clipboard
| Challenge: | a new technique for layering explicit pragmatic inference on top of models for sequential tasks is proposed . explicit pragmatic reasoning is used to generate and follow natural language instructions . |
| Approach: | They propose a pragmatic speaker that uses the base listener to simulate the interpretation of candidate descriptions and a listener that reasons counterfactually about alternative descriptions. |
| Outcome: | The proposed model improves state-of-the-art models for interpreting human instructions and speaker models in diverse settings. |
Spatially-Aware Speaker for Vision-and-Language Navigation Instruction Generation (2024.acl-long)
Copied to clipboard
| Challenge: | Existing speaker models learn strategies to evade evaluation metrics and obtain higher scores even for low-quality sentences. |
| Approach: | They propose a speaker-based instruction generator that utilises both structural and semantic knowledge of the environment to produce richer instructions. |
| Outcome: | The proposed model outperforms existing models and is evaluated using standard metrics. |